We can write optimizer: PySwarms class in a modular format: optimizer : pyswarms.single . This already defines both pyswarms.single.LocalBestPSO and pyswarms.single.GlobalBestPSO 😄

Got it ✅

Even if this is not a user-facing method, maybe we can add docstrings here? Just for others who would look at the code. 👍

"""Generates score for optimizer's performance on objective function.

Parameters
-------------

options : dict
    <my description>
"""

The same goes for other methods 😄

Got it ✅

I think it is fine even if we remove the else construct because the min serves as the
default behaviour. Can we confirm this based on your unit tests?

# Catches that maximum bool flag
 if maximum:
    idx, self.best_score = max(enumerate(scores), key=op.itemgetter(1))

# Default behaviour
idx, self.best_score = min(enumerate(scores), key=op.itemgetter(1))

Cool. I set this up to first run the default assignments, and then reassign if the maximum bool flag is caught (just flipping the order of what you show here). 🔄

Hmmm... I may be wrong or my GitHub is not working properly, but does this need to have an __init__ method similar to RandomSearch()? 😕 😄

You also mentioned about the possibility that GridSearch will take so long when we're looking for a lot of hyperparameters and take too many iterations 👍 . Maybe we can add a logger.warn? check the implementations in GlobalBestPSO and LocalBestPSO.

There is a logger.info('My information') call in place of print() functions. I also initialized it with logger = logging.getlogger(__name__). You can do the same thing but call logger.warn() instead.

Thus we can have something like:

if condition_is_met:
    logger.warn('Using GridSearch will take so much time') 
# of course you can write a better warning than mine hehe

This will still run a search, only with an additional warning.

Ha! Yes, I think I thought I was being clever because if no new attributes are added in the GridSearch __init__ it could just just inherit SearchBase's __init__, which would be retrieved when creating a GridSearch instance. But looking at it now, I think it's clearer to call the super().__init__ explicitly.

+  57     def __init__(self, optimizer, n_particles, dimensions, options,
+  58                  objective_func, iters,bounds=None, velocity_clamp=None):
+  59         """Initializes the paramsearch."""
+  60
+  61         # Assign attributes
+  62         super().__init__(optimizer, n_particles, dimensions, options,
+  63                 objective_func, iters, bounds=bounds,
+  64                 velocity_clamp=velocity_clamp)

Thanks for pointing me to 'logger.warning'. In looking at search I realized I'm not sure how much of an issue runtime will be, even for GridSearch with lots of hyperparameter options. My hunch was that runtime on big search spaces would need to be flagged for users, but I don't have a good sense of when a user would input a really long list for each of the 'c' weights, for instance.

What I'd like to do next is write up an example for the docs on Tuning PSO hyperparameters that shows these two methods in action, and use that writing as a chance to read the literature on conventions for setting PSO hyperparameters. I think this will help me have a better sense of kinds of situations might come up. 📚

What I can see happening is including logger.warn, but also having clear documentation that speaks to the conventions for setting options (and perhaps resources linked in the Example) that would steer users away from setting search values that don't quite make sense, and might take too long to run.

Actually, I realized that it's kinda hard to pinpoint when to issue a warning based on user input. I believe your plan is good for now 👍

Regarding the __init__, the reason why I often add the super() is that it enables me to pass defaults into the base class. So just in case I want defaults in GridSearch etc., I just declare them on GridSearch's init, then do the super. But for now i guess this implementation is fine